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(54) System and method for adjusting the convergence of a stereo camera 



(57) In a correction operating part (10c), three re- 
gions including two distant regions and one near region 
are selected on a reference image picked up by a main 
camera (2), and distance data are added to the positions 
of the respective regions to define a range, in which cor- 
responding regions exist, in a comparative Image pick 
up by a sub-camera (3) to search the range to determine 
the positions of the regions at a resolution of one pixel 
or less. Then, on the basis of one of the distant regions, 
a translation correction vale and an angle of rotation for 
the comparative image are derived from the relationship 



between the positions of the respective regions to be 
added to the set value of an affine transforming circuit 
(18) to parallel-displace and rotate the comparative im- 
age. Thereafter, an angle of rotation for causing the hor- 
izontal line of the main camera (2) to be parallel to the 
base line of a stereo camera is derived to be added to 
the set value of an affine transforming circuit (17) to ro- 
tate the reference image. Thus, the deviation of the op- 
tical axis of the stereo camera (1 ) is automatically ad- 
justed while carrying out a usual operation for picking 
up an image of an ordinary landscape to carry out the 
stereo image processing (40). 
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Description 

BACKGROUND OF THE INVENTiON 
^ Field of the Invention 

[0001] The present invention relates generally to a stereo camera systenn and method for adjusting the deviation of 
an optical axis of a stereo camera. 

Related Background Art 

[0002] As a three-dimensional measuring technique based on images, there is known an image processing based 
on a so-called stereo method for deriving a correlation between a pair of images, which are obtained by picking up an 
object from different positions by two cameras (a stereo camera), to derive a distance by a parallax for the same object 
on the basis of the principle of triangulation using camera parameters, such as the mounting position and focal length 
of the stereo camera. 

[0003] In this image processing based on the stereo method, two picture signals obtained by the stereo camera are 
sequentially shifted to be superposed on each other to derive a position at which the two picture signals are coincident 
with each other Therefore, it is desired that there are no deviations other than the parallax between the two picture 
signals, so that the optical positioning of the stereo camera is important. 

[0004] Thus, Japanese Patent Laid-Open No. 5-157557 discloses that a holding member for connecting and holding 
a pair of video cameras has parallelism adjusting means for adjusting the arrangement of pixels of an image sensor 
of one of the video cameras so as to be parallel to the arrangement of pixels of an image sensor of the other video 
camera, and an optical axis adjusting member for adjusting the optical axis of one of the video cameras so as to be 
parallel to the optical axis of the other video camera, so that the correlation between the two cameras is mechanically 
adjusted and held. 

[0005] However, even if the stereo camera is fixed once, the deviation of the stereo camera is easily caused with 
secular change. Conventionally, if such a deviation is caused, the stereo camera must be mechanically readjusted. 
Therefore, it is not only required to carry out the complicated operation, but it also takes a lot of time to readjust the 
stereo camera, so that there is a limit to the mechanical adjustment to ensure required accuracy 
[0006] In order to cope with this problem. Japanese Patent Application No. 9-11 7268. which is assigned to the present 
Assignee and incorporated herein by reference, discloses a technique for electrically adjusting the optical position of 
a stereo camera without the need of any mechanical adjustments. This technique can accurately adjust the optical 
position of a stereo camera to a level, at which it is difficult to mechanically adjust the optical position of the stereo 
camera, and easily readjust the deviation due to secular change after the adjustment. 

[0007] However, since the technique disclosed in Japanese Patent Application No. 9-117268 is designed to pick up 
an image of a special adjusting pattern by means of a stereo camera to correct the optical position of the stereo camera. 
It fs required to periodically stop the operation of the stereo camera in an image processing field to adjust the optical 
position of the stereo camera, so that it is required to care maintenance. 

SUMMARY OF THE INVENTION 
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[0008] It is therefore an object of the present invention to eliminate the aforementbned problems and to provide a 
stereo camera adjusting system and method capable of automatically adjusting the age-based deviation of the optical 
axis of a stereo camera while maintaining the operating state thereof from an early stage, in which it is possible to 
disregard the Influence on ranging accuracy. 

[0009] In order to accomplish the aforementioned and other objects, according to one aspect of the present invention, 
there is provided a stereo camera adjusting system for adjusting an optical axis of a stereo camera having first and 
second cameras arranged at regular inten/als, the system comprising: image adjusting means for geometrbafly trans- 
forming an image, which is picked up by the second camera, by a translation correction and a rotation correction, and 
for geometrically transforming an image, which is picked up by the first camera, by a rotation correction; and correction 
operating means for calculating a difference between corresponding positions of the images, which are picked up by 
the first and second cameras, on the basis of distance data obtained by stereo-processing the pair of images picked 
up by the first and second cameras, and for setting a transforming value for the image transformation so as to remove 
^5 the difference. 

[0010] The correction operating means may calculate the difference on the basis of two distant regions substantially 
at the same distance and one near region. The correction operating means may define a range including regions 
corresponding to the regions, in the image picked up by the second camera, on the basis of distance data of the distant 
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regions of the image picked up by the first camera, and search the defined range to derive the distant regions of the 
image picked up by the second camera. The image adjusting means may geometrically transform the image, which is 
picked up by the second camera, by a horizontal translation correction and a vertical translation correction, and the 
correction operating means may calculate a transforming value for the image transformation based on the horizontal 
translation correction for the image picked up by the second camera, on the basis of a deviation between the positions 
of the regions of the images picked up by the first and second cameras, and distances of the regions. The correction 
operating means may set the transforming value for the image transformation on the basis of a pair of images of a 
landscape picked up by the first and second cameras. 

[0011] According to another aspect of the present invention, there is provided a stereo camera adjusting method for 
adjusting an optical axis of a stereo camera having first and second cameras provided at regular intervals, the method 
comprising the steps of: calculating a difference between corresponding positions of images picked up by the first and 
second cameras, on the basis of distance data obtained by stereo-processing the pair of images picked up by the first 
and second cameras; setting a translation correction value and a rotation correction value for the second camera so 
as to remove the calculated difference; and geometrically transforming the image picked up by the second camera, by 
a translation correction and a rotation correction, on the basis of the set translation correction value and the set rotation 
correction value for the second camera. 

[0012] The stereo camera adjusting method may further comprise the steps of: setting a rotation correction value 
for the first camera so as to remove the calculated difference; and geometrically transforming the image picked up by 
the first camera, by a rotation correction on the basis of the set rotation correction value for the first camera, 
[0013] That is. according to the present invention, on the basis of distance data obtained by processing a pair of 
images of a landscape picked up by a stereo camera, a difference between corresponding positions of images picked 
up by first and second cameras is calculated. Then, an image transforming value for a rotation correction in the image 
picked up by the first camera, and image transforming values for a translation correction and a rotation correction in 
horizontal and vertical directions in the image picked up by the second camera are set so as to remove the difference. 
Then, the images picked up by the first and second cameras are geometrically transformed to automatically correct 
the age-based deviation of the optical axis of the stereo camera. 

[0014] Thus, since the age-based deviation of the optical axis of the stereo camera is automatically adjusted while 
maintaining the operating state thereof from an early stage, in which it is possible to disregard the influence on ranging 
accuracy, it is possible to always use the stereo camera in an accurately adjusted state, and it is not required to stop 
the operation of the stereo camera in a field unlike the technique for picking up an image of a special adjusting pattem 
to adjust the optical position of the stereo camera, so that there are excellent advantages in that it is possible to achieve 
maintenance free. 

[0015] The difference between the corresponding positions is preferably calculated on the basis of two distant re- 
gions, which are arranged substantially at the same distance, and one near region. Thus, it is possible to simultaneously 
carry out corrections for relative positions of the respective images and a correction for causing the horizontal line of 
the first camera to be parallel to the base line of the stereo camera. 

[0016] In addition, the positbns of the distant regions of the image picked up by the second camera may be obtained 
by adding distance data to the positions of the distant regions of the image picked up by the first camera, defining a 
range including corresponding regions, in the image picked up by the second camera, and searching the defined range. 
The horizontal translation correction for the image picked up by the second camera may be calculated on the basis of 
the deviations of the positions of the respective regions in the images picked up by the first and second cameras, and 
the distances of the respective regions. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0017] A preferred embodiment of the present invention will become understood from the foltowing detailed descrip- 
tion referring to the accompanying drawings. ^ ' 
[0018] In the drawings: 

FIG. 1 is a block diagram of a preferred embodiment of a stereo camera adjusting system according to the present 
invention; 

FIG. 2 is a block diagram showing the detail of an image adjusting part (an affine transforming circuit) of FIG. 1 ; 
FIGS. 3A and 3B are schematic diagrams showing the whole constructions of a stereo camera adjusting system 
according to the present invention; 

FIG. 4 is a timing chart showing the image capturing timing and affine transforming timing in the image adjusting 
part of FIG. 1; 

FIG. 5 is a flow chart of an image correcting routine in a correction operating part of FIG. 1 ; 

FIG. 6 is a schematic diagram for explaining corresponding regions between a reference image and a comparative 
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image; 

FIG. 7 is a schematic diagram for explaining an angle of rotation in the affine transformation of a comparative image; 
FIG. 8 is a schematic diagram for explaining the deviation of a horizontal line of a main camera from a base line; and 
FIG. 9 is a schematic diagram for explaining a camera system viewed from side. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

[0019] Referring now to the accompanying drawings, the preferred embodiment of the present invention will be de- 
scribed below. FIGS. 1 through 9 show the preferred embodiment of the present invention. 

[0020] First, referring to FIGS. 3A and 3B. the whole construction of the preferred embodiment of the present invention 
will be described. Furthermore, FIGS. 3A and 38 are top and side views of a stereo camera, respectively. In FIGS. 3A 
and 38, a stereo camera 1 comprises two cameras 2 and 3 synchronized with each other Each of the cameras 2 and 
3 is provided with an image sensor, such as a charge coupled device (CCD), therein and capable of changing the 
shutter speed thereof. One camera (first camera) 2 serves as a main camera for picking up a reference image for a 
stereo processing, and the other camera (second camera) 3 serves as a sub-camera for picking up a comparative 
image for the stereo processing. These cameras 2 and 3 are provided on a stay 4 at regular intervals. 
[0021] Two innages picked up by the cameras 2 and 3 are inputted to a stereo processing unit 40 via an image input 
unit 10. In the stereo processing unit 40, the conformity of the two images with each other is estimated by the stereo 
matching to calculate a three-dimensional position of an object, to recognize an image of the object, and so forth. In 
this case, the two cameras 2 and 3 are adjusted to be fixed on the stay 4 so that the optical axes thereof are parallel 
to each other at an initial manufacturing stage. However, there are the possibilities that the deviations of the optical 
axes are gradually caused in usual use, and the ranging accuracy is gradually lowered in the long run, so that errors 
can not disregarded finally. 

[0022] For that reason, the image input unit 10 provided between the cameras 2, 3 and the stereo processing unit 
40 has the function of adjusting the optical positions of the cameras 2 and 3 by an image correction, to automatically 
correct the deviations of images picked up by the cameras 2 and 3 to precisely and equivalently adjust the optical axes 
of the cameras 2 and 3 in a stage that the deviations of the optical axes of the cameras 2 and 3 are minor, i.e., in an 
early stage that mismatching is small during the stereo matching in the stereo processing unit 40. 
[0023] Referring now to FIG. 1 , the detail of the image input unit 1 0 of FIG. 3 will be described below 
[0024] As shown in FIG. 1, the image input unit 10 comprises: an image input part 10a for inputting a picked-up 
image; an image adjusting part 1 0b for carrying out a geometrical transfonmation, such as rotation and parallel dis- 
placement of the image; and a correctbn operating partJOc for calculating an transforming value for the image. 
[0025] The inr>age input part 10a has analog interfaces 11 and 12 for adjusting analog innages, which are outputted 
from the cameras 2 and 3, to downstream input ranges, and A/D converters 13 and 14 for transforming the analog 
images into digital images having predetermined brightness and gradation (e.g.. a gray scale of 256 gradations). 
[0026] The innage adjusting part 1 0b has input image memories 1 5 and 1 6 for temporarily storing the digitized images, 
and affine transforming circuits 17 and 18 for carrying out a geometrical transformation, such as rotation and parallel 
displacement, of the digital images stored in the input inoage memories 15 and 16 to output the geometrically trans- 
formed images to the stereo processing unit 40. 

[0027] The affine transforming circuits 1 7 and 1 8 carry out corrections when the corresponding points of the images 
picked up by the cameras 2 and 3 move geometrically from the original positions due to the deviations of the optical 
axes. As a representative one of the affine transforming circuits 1 7 and 1 8, the internal construction of the affine trans- 
forming circuits 17 is shown in FIG. 2. 

[0028] The affine transfomnihg circuit 1 7 comprises: an image memory data interface 20 for writing data into the input 
image memory 15 and for reading data out of the input image memory 15; an image memory address interface 21 for 
addressing the input image memory 1 5; an input image writing address generating circuit 22 for generating an address 
when the image data outputted from the A/D converter 1 3 are written in the input image memory 15; an inverse affine 
transformation reading address generating circuit 23 for generating an address, which is used for reading the image 
data out of the input image memory 1 5 to geometrically transform the image, by the inverse affine transformation: and 
an interpolation operating circuit 24 for carrying out the linear interpolation of data, which have been read by the inverse 
affine transformation, to output transformed image data, 

[0029] That is, in the geometrical transformation of the image using the affine transforming circuits 17 and 18, the 
original image before transfomDation and the transformed image are digital images wherein pixels are arranged in the 
form of a grid, and the concentration values of the pixels on the transformed image are given by the concentration 
values of the corresponding pixel positions derived by the inverse affine transformation of the original image. The 
corresponding pixel positions derived by the inverse affine transformation are not generally integer pixel positions, so 
that the pixels on the transformed image do not correspond to any pixels on the original image. Therefore, the inter- 
polation operating circuit 24 carries out the linear interpolation using the concentration values of four surrounding pixels, 



EP0 942 610 A2 



to closely derive the concentration values of the pixels on the transformed image. 

[0030] As shown in FIG. 4, when image signals outputted from the cameras 2 and 3, e.g., image signals, such as 
NTSC picture signals, outputted from the cameras 2 and 3 in synchronism with a line synchronizing signal and a frame 
synchronizing signal, are sampled in a sampling period of a field signal, the affine transforming circuits 17 and 18 
transform the sampled image in the next field interval. 

[0031] That is, the gain offset and so forth of image signals outputted from the cameras 2 and 3 are adjusted to the 
input ranges of the A/D converters 13 and 14 by means of the analog interfaces 11 and 12. The digital image data 
converted by the A/D converters 13 and 14 are stored in the input image memories 15 and 16 in accordance with 
addresses generated by the input image writing address generating circuits 22 and 22 of the affine transforming circuits 
17 and 18. In the next field inten/al, the concentration data at addresses generated by the inverse affine transformation 
reading address generating circuits 23 and 23 of the affine transforming circuits 17 and 18 are read out of the input 
image memories 1 5 and 16. Then, the interpolation operations of the concentration data are carried out by means of 
the interpolation operating circuits 24 and 24 of the affine transforming circuits 17 and 18 to output the transformed 
images to the stereo processing unit 40. 

[0032] The correction values for the variations of optical axes of the cameras 2 and 3, i.e.. the image transforming 
values in the affine transforming circuit 17 and 1 8, can be obtained from coordinate data of a coincident point measured 
from a picked-up image, which is obtained by picking up a special adjusting pattern arranged at a known distance (a 
distance by which a parallax can be inversely calculated). However, according to the present invention, while an image 
of an ordinary landscape is picked up to carry out the usual operation for the stereo image processing without the need 
of any special adjusting pattems, the correction values are operated by the correction operating part 1 0c to be fed lo 
the affine transforming circuits 17 and 18 as the image transforming values to autonnatically make an adjustment. 
[0033] The correction operating part .10c comprises a computer having a CPU 31 , image memories 32 and 33 and 
so forth. The correction operating part 1 0c is connected to the A/D converters 1 3 and 1 4 of the image input unit 10 via 
the image memories 32 and 33, and to the stereo processing unit 40. It is noted that the correction operating part 1 0c 
may be a part of the stereo processing unit 40 and carry out the correction processing as a background job of the 
stereo image processing. 

[0034] Referring to a flow chart of FIG. 5, the image correction carried out by the image adjusting part 10b and the 
correction operating part 10c will be described below. 

[0035] FIG. 5 is an image correcting routine executed by the CPU 31 of the correction operating part 10c every a 
predetermined period. In this routine, at step SI 00, the results of a group filter, which are obtained when images of a 
landscape picked up by the cameras 2 and 3 are processed to extract three-dimensbnal object data from a distance 
image, are read out of the stereo processing unit 40, and some small regions for operating correction values are set 
in the image picked up by the main camera 2, I.e.. in a reference image. 

[0036] For example, it is assumed that each of the small regions is a region of 16 x 16 pixels, the half or more of 
which include corresponding distance image data. As the small regions, three regions including two distant regions at 
a long distance of about 50 to 60 m and one near region at a short distance of up to about 5 m are selected. The two 
distant regions are spaced from each other so as to have substantially the same distance data. Furthermore, the group 
filter is a software filter for detecting a group having an image deviation similar to those of surrounding pixels to efficiently 
remove noises included in the distance image. Such a group filter is disclosed in Japanese Patent /Application No. 
9-86877, which is assigned to the present Assignee and incorporated herein by reference. 

[0037] Then, the routine goes to step S1 10 wherein distance data (deviation) are added to the positions of the re- 
spective regions of the reference image set at step S100, to define a range for searching the corresponding regions 
in the image picked up by the sub-camera 3, i.e., in the comparative image. Then, at step S1 20, this range is searched 
to derive the positions of the regions at a resolution of one pixel or less (e.g., a resolution of 1/10 pixels) by the sub- 
pixel processing. 

[0038] As a result, as shown in FIG. 6, in a coordinate system wherein the origin is arranged at a left-upper comer 
of an image and X and Y coordinates are arranged in horizontal and vertical directions, the position coordinates (Xcl , 
Yd) and (Xc2, Yc2) of two distant regions # 1C and # 2C and the position coordinate (Xc3, Yc3) of one near region 
U3C of the comparative image are derived for the position coordinates (Xrl , Yrl ) and (Xr2, Yr2) of two distant regions 
#1 R and #2R and the position coordinate (Xr3, Yr3) of one near region #3R of the reference image. 
[0039] Subsequently, at step SI 30. a difference between the Y components of the position coordinates of one of the 
two distant regions on the reference image side and the corresponding distant region on the comparative image side, 
e.g., a difference (Yrl -Yd ) between the Y components of the position coordinate (Xrl . Yrl ) of the left distant region 
# 1 R on the reference image side and the position coordinate (Xcl , Yd ) of the corresponding left distant region # 1C 
on the comparative image side in FIG. 6. is derived to be added to the set value of the affine transforming circuit 18 
on the comparative image side as a Y direction translation correction value, i.e.. as an image translating transformation 
value in Y directions, to parallel-displace the image (the comparative image) of the sub-camera 3 in Y directions. 
[0040] Then, the routine goes to step SI 40 wherein an X direction translation correction value for the comparative 
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image is derived to be added to the set value of the affine transforming circuit 18 on the comparative image side to 
parallel-displace the image of the sub-camera 3 in X directions. This X direction translation correction value can be 
given by a deviation ZO between the reference and comparative images at infinity. The deviation 20 at infinity can be 
represented by the following formula (1 ) using a distance d1 to the distant region, a distance d3 to the near region, a 
deviation 21 between the reference and comparative images in the distant region, and a deviation 23 between the 
reference and comparative images in the near region. Thus, it is assumed that -20 is a translation amount of the 
comparative image in lateral directions (X directions). 
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20 = (d3 • 23-d1 • 21 )/(d1 -d3) (1 ) 



[0041] Originally, at the corresponding points of the reference and comparative images, only X coordinates in hori- 
zontal directions must be different to detect a parallax, and Y coordinates must be the same. Therefore, at the above 
step SI 30, the positions of the Y coordinates of one of the two distant regions in the reference and comparative images 
^-5 are caused to be coincident wit each other, and at the above step SI 40, the position of the X coordinate in the reference 
image is adjusted to a position, at which the original parallax is to be detected. 

[0042] Then, at step SI 50, the comparative image is rotated so that the position of the Y coordinate of the other 
region of the comparative image is coincident with that of the reference image. That is, as shown in FIG. 7, the com- 
parative image is rotated about the position coordinate (Xr1 , Yrl ) of the left distant region # 1 R of the reference image 
20 so that the Y coordinate value Yc2 of the right distant region # 2C of the comparative image is coincident with the Y 
coordinate value Yr2 of the right distant region # 2R of the reference image. When the coordinate (Xc2, Yc2) is rotated 
about the coordinate (Xrl , Yrl ) by an angle of rotation 0. a coordinate (Xc2', Yc2') after rotation can be represented 
by the following formula (2). 

/Afc2'\ _ /cos-f> -sin\^>\/Ac2- Ayi\ f Xr\\ 

\ YcT) ^ {sm-ty cosi^> Jv Yc2 - Yrl )\Yr\] ^ ^ 



[0043] If the Y coordinate value Yc2' of the region # 2C of the comparative innage after rotation is coincident with the 
Y coordinate value Yr2 of the region # 2R of the reference image, and the above formula (2) is rearranged with respect 
to only Y components assuming that Yc2' = Yr2. the following formula (3) can be obtained. 



35 Yr2-Yr1 = (Xc2-Xr1 ) • sin 6 + (Yc2-Yr1 ) • cos0 (3) 

[0044] Moreover, if Yr2-Yr1 = A. Xc2-Xr1=B, Yc2-Yr1=C, and cos 9 =±(1-sin2e)i^ are substituted for the above 
formula (3) to solve the formula (3) with respect to sin 0, the following formula (4) can be obtained. 

40 

2 2 2 1/2 2 2 

Sin 0 = (A • B ± C • (B +C -A ) )/(B -i-C*^) (4) 

[0045] When Yr2 =Yc2 and A=C, i.e., when the Y coordinate value of the right distant region # 2R of the reference 
image has been coincident with the Y coordinate value of the right distant region # 2C of the comparative image, the 
above formula (4) is 0 since rotation is not required. In this case, it can be seen from the following formula (5) that the 
sign of the second term of the numerator of the above formula (4) must be negative. 

sin G = B*(A±C)/(B^+C^) = 0 ^ (5) 

so 

[0046] Therefore, the angle of rotation Ocan be derived by the following formula (6). This angle of rotation G is added 
to the set value of the affine transforming circuit 18 on the comparative image side as an image rotating transformation 
value about the coordinate (Xrl , Yrl ) to rotate the image (the comparative image) of the sub-camera 3. 

55 

0 = sin'"* (A • B-C • (B^+C^-A^)^^)/(B^+C^) (6) 
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[0047] By this rotation of the comparative image, the near region # 3C of the comparative image is also rotated. 
When the base lines of the cameras 2 and 3 are not parallel to the horizontal line of the main camera 2 so that the 
horizontal line of the main camera 2 is inclined by an angle ^ with respect to the base lines as shown in FIG. 8. there 
is a difference AYm - AYs between the Y coordinate value YrS of the near region # 3R of the reference image and the 
Y coordinate value Yc3 of the near region #3C of the comparative innage after rotation as shown in FIG. 9. 
[0048] Therefore, at the next step S160. the rotation by the angle of rotation ({> about the coordinate (Xr1. Yrl) on 
the reference image side is added to the set value in order to remove the difference AYm - AYs. As can be seen clearly 
from FIG. 6, the angle of rotation <^ can be derived from the base length B of the canneras 2 and 3 and a deviation AY 
between the centers of the reference and comparative images on a focal plane. The deviation AY can be derived using 
the focal point f. the distance d1 to the distant region, the distance d3 to the near region and the difference AYm - AYs 
on the basis of the geometrical relationship when the camera system is viewed fronn side as shown in FIG. 9. 
[0049] That is. the deviation AY can be derived by the following formula (7) using the difference AYm - AYs between 
the deviation A Ym of the near region from the center of the reference image fornied on the CCD surface (image picked- 
up surface) and the deviation AYs of the near region fi-om the center of the comparative image formed on the CCD 
surface. From the deviation AY and the base length B. the angle of rotation (t> can be finally derived by the following 
formula (8). 

AY = (AYm-A Ys) • d1 • d3/(f .(d1 -d3)) (7) 

0 = tan-'(AY/B) 

= tan- '((AYm -AYs) -dl • ci3)/(B - f -(dl-dS)) (8) 

[0050] When the comparative and reference images are rotated in the above manner, the routine goes from the 
Hbovo step S160 to step S170 wherein it is examined whether the angles of rotation 9 and (t> are sufficiently small to 
ho within a range of allowable values. When the angles of rotatbn 9 and <^ have not reached the allowable values, the 
rot jt inn returns to the step S120 wherein the position of the corresponding region of the comparative image is deter- 
mined by the sub-pixel processing again. Then, the same processing is repeated, and when the angles of rotation 9 
find t> rifo sufficiently small below the allowable value, the routine ends. 

(0051) Thus, the deviations of the optical axes of the cameras 2 and 3 gradually caused in usual use can be auto- 
mHtiCrtlly corrected while maintaining the operation state in the field. 

[0052] While the presently preferred embodiment of the present invention has been shown and described, it is to be 
understood that this disclosure is for the purpose of illustration and that various changes and modifications may be 
m-idc wihout departing from the scope of the invention as set forth in the appended claims. 

Claims 

1. A sloreo camera (1) adjusting system for adjusting an optical axis of a stereo camera having first and second 
cameras (2, 3) arranged at regular intervals, comprising: 

image adjusting means (1 0, 10b) for geometrically transforming an Image picked up by said second camera 
(3) by a translation correction and a rotation correction and for geometrically transfomning an image picked 
up by said first camera by a rotation correction; and 

correction operating means (10c) for calculating a difference between corresponding positions of said images 
picked up by said first and second cameras (2, 3) on the basis of distance data obtained by stereo-processing 
(40) the pair of innages picked up by said first and second cameras (2, 3) and for setting a transforming value 
for said image adjusting means (10) so as to remove said difference. 

2. The stereo camera (1) adjusting system as set forth in Claim 1. wherein said correction operating means (10c) 
calculates said difference on the basis of twodistant regions substantially at the same distance and one near region. 

3. The stereo camera (1) adjusting system as set forth in Claim 2, wherein said correction operating means (10c) 
defines a range in said image picked up by said second camera (3) on the basis of distance data of said distance 
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regions of said image picked up by said first camera (1), and searches the defined range to derive said distant 
regions of said image picked up by said second camera (3). 

The stereo camera (1 ) adjusting system as set forth in Claim 2, wherein said image adjusting means (10b) geo- 
metrically transforms said image, which is picked up by said second camera (3). by a horizontal translation cor- 
rection and a vertical translation correction, and wherein said correction operating means (10c) calculates a trans- 
forming value for said horizontal translation correction for said image picked up by said second camera (3), on the 
basis of a deviation between the positions of said regions of said images picked up by said first and second cameras 
(2, 3), and distances of said regions. 

The stereo camera (1) adjusting system as set forth in Claim 1, wherein said correction operating means (10c) 
sets said transforming value for said image adjusting means (10b) on the basis of a pair of images of a landscape 
picked up by said first and second cameras (2. 3). 

A stereo camera (1) adjusting method for adjusting an optical axis of a stereo camera having first and second 
cameras (2, 3) provided at regular intervals, comprising the steps of: 

calculating a difference between corresponding positions of images picked up by said first and second cameras 
(2, 3) on the basis of distance data obtained by stereo-processing (40) the pair of images picked up by said 
first and second cameras (2, 3); 

setting a translation correction value and a rotation correction value for said second camera (3) so as to remove 
the calculated difference; and 

geometrically transforming said image picked up by said second camera (3) by a translation correction and a 
rotation correction on the basis of the set translation correction value and the set rotation correction value for 
said second camera (3). 

The stereo camera (1 ) adjusting method as set forth in Claim 6, which further comprises the steps of: 

setting a rotation correction value for said first camera so as to remove the calculated difference; and 

geometrically transforming said in^ge picked up by said first camera (2), by a rotation correction on the basis 
of the set rotation correction value for said first camera (2). 
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(54) System and method for adjusting the convergence of a stereo camera 

(57) The present invention relates generally to a 
stereo camera systenn and method for adjusting the de- 
viation of an optical axis of a stereo camera. In a cor- 
rection operating part (10c), three regions including two 
distant regions and one near region are selected on a 
reference image picked up by a main camera (2), and 
distance data are added to the positions of the respec- 
tive regions to define a range, in which corresponding 
regions exist, in a comparative image pick up by a sub- 
camera (3) to search the range to determine the posi- 
tions of the regions at a resolution of one pixel or less. 
Then, on the basis of one of the distant regions, a trans- 
lation correction vale and an angle of rotation for the 
comparative image are derived from the relationship be- 
tween the positions of the respective regions to be add- 
ed to the set value of an affine transforming circuit (18) 
lo parallel-displace and rotate the comparative image. 

Thereafter, an angle of rotation for causing the horizon- PIG^ 1 

tal line of the main camera (2) to be parallel to the base 
line of a stereo camera is derived to be added to the set 
value of an affine transforming circuit (17) to rotate the 
reference image. Thus, the deviation of the optical axis 
of the stereo camera (1 ) is automatically adjusted while 
carrying out a usual operation for picking up an image 
of an ordinary landscape to carry out the stereo image 
processing (40). 
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